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1.  Introduction 


This  paper  addresses  the  following  linearly  constrained  nonlinear  pro¬ 
gramming  problem: 

P:  minimize  f(x) 
subject  to  Ax  b 

where  f  is  a  twice  continuously  differentiable  function  on  Rnt  and  A  is  an 
txn  matrix  whose  jth  row  is  denoted  by  a^,  and  where  a  superscript  t  denotes 
the  transpose  operation. 

There  are  several  approaches  for  solving  this  problem.  The  first  one 
relies  on  partitioning  the  variables  into  basic,  nonbasic,  and  superbasic 
variables.  The  values  of  the  superbasic  and  basic  variables  are  modified 
while  the  nonbasic  variables  are  fixed  at  their  current  values.  Examples  of 
methods  in  this  class  are  the  convex  simplex  method  of  Zangwill  [18],  the 
reduced  gradient  method  of  Wolfe  [17],  the  method  of  Murtagh  and  Saunders  [12], 
and  the  variable  reduction  method  of  McCormick  [8]* 

Another  class  of  methods  is  the  extension  of  quasi-Newton  algorithms  from 
unconstrained  to  constrained  optimization.  Here,  at  any  iteration,  a  set  of 
active  restrictions  is  identified,  and  then  a  modified  Newton  procedure  is 
used  to  minimize  the  objective  function  on  the  manifold  defined  by  these  active 
constraints.  See  for  example  Goldfarb  [6],  and  Gill  and  Murray  [5]. 

Other  approaches  for  solving  problems  with  linear  constraints  are  the 
gradient  projection  method  and  the  method  of  feasible  directions.  The  former 
computes  a  direction  by  projecting  the  negative  gradient  on  the  space  ortho¬ 
gonal  to  the  gradients  of  a  subset  of  the  binding  constraints  while  the  latter 
method  determines  a  search  direction  by  solving  a  linear  programming  problem. 
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For  a  review  of  these  methods  the  reader  may  refer  to  Rosen  [14],  Zoutendljk 
[19],  Frank  and  Wolfe  [4],  and  Topkis  and  Veinott  [15], 

In  this  paper,  an  algorithm  for  solving  problem  ?  is  proposed*  At  each 
Iteration  a  correction  vector  is  computed  by  finding  the  minimum  distance 
from  a  given  point  to  a  polyhedral  cone  defined  in  terms  of  the  gradients 
of  the  "almost”  binding  constraints*  An  approximate  line  search  procedure 
which  extends  those  of  Armijo  [1]  and  >fukai  [10,  11]  for  unconstrained  opti¬ 
mization  is  developed  for  determining  the  step  size.  First,  an  estimate  of 
the  step  size  based  on  a  quadratic  approximation  to  the  objective  function  is 
computed,  and  then  adjusted  if  necessary. 

In  Section  2,  we  outline  the  algorithm.  In  Section  3,  we  show  that 
accumulation  points  of  the  algorithm  are  Kuhn-Tucker  points  to  a  slight  per¬ 
turbation  of  the  original  problem.  Finally,  in  Section  4,  assuming  that  the 
algorithm  converges,  and  under  suitable  second  order  sufficiency  optimality 
conditions,  we  show  that  the  step  size  estimates  which  are  based  on  the  quad¬ 
ratic  approximation  are  adceptable  so  that  only  one  functional  evaluation  is 
eventually  needed  for  performing  the  line  search. 

2.  Statement  of  the  Algorithm 
Consider  the  following  algorithm  for  solving  Problem  P* 

Step  0 

Choose  values  for  the  parameters  c,  z,  6,  and  e.  Select  a  point  such  that 
AXq  £  b  and  let  5Q  *  6.  Let  i  •  0  and  go  to  Step  1. 

Step  1 

Let  w^  be  the  optimal  solution  to  Problem  D(x^)  given  below; 


and  let  *  (S^,  and  go  to  Step  4.  Otherwise,  let  •  1,  6^, 

and  go  to  Step  4. 

Step  4 
Let 

“i  =  {M^ 

and  compute  the  smallest  nonnegative  integer  k  satisfying 
f  (xi+(j)ka1di)  -  f^)  £  j^^Vf  (xi)td1 

1  ki 

Let  k^  *  k,  x^+2  “  x±  +  i  *  i  +  1,  and  go  to  Step  1. 

The  following  remarks  are  helpful  in  interpreting  the  above  algorithm* 

1.  A  direction  is  determined  by  solving  Problem  D(xi)*  This  problem 

finds  the  point  in  the  convex  polyhedral  cone  {w:  a^w  <_  0  for  jeKx^)} 

which  is  closest  to  the  vector  -  —  7f(x.).  Methods  of  least  distance  pro- 

2  1 

gramir.ing,  as  in  the  works  of  Bazaraa  and  Goode  [2],  and  Wolfe  [16]  can  be 
used  for  solving  this  problem*  Special  methods  that  take  advantage  of  the 
structure  of  the  cone  constraints  may  prove  quite  useful  in  this  regard. 


(2.7) 


(2.8) 


2.  The  restrictions  enforced  in  Problem  D (x^)  are  the  c-binding  constraints 
at  that  is,  those  satisfying  bj  -  c  <  **  wi  “  0»  tk* 

algorithm  is  terminated  with  x^.  In  thiB  case,  from  the  Kuhn-Tucker  condi¬ 
tions  for  Problem  DCx^),  there  exist  for  jel(x^)  such  that: 
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Vf(x.)  +  7  u.a  *  0 

1  jeKxp  3  J 

Uj  2.  0  for  JsKxj) 

These  conditions  imply  that  is  a  Kuhn-Tucker  point  for  the  following 
problem: 

minimize  f (x) 

t  t 

subject  to  a^x  £  a^xi  for  jelCx^) 

ajX  —  bj  f°r 

Noting  that  -  c  <  £  b_.  for  jel(x^),  if  c  is  sufficiently  small,  it 

is  clear  that  the  algorithm}  is  terminated  if  is  a  Kuhn-Tucker  solution  to 
a  slightly  perturbed  version  of  Problem  P.  The  following  definition  will 
thus  be  useful. 

Definition  2.1 
* 

Let  x  be  a  feasible  point  to  Problem  P.  If  the  optimal  solution  to  Problem 
D(x  )  is  equal  to  zero,  then  x  is  called  a  c-KT  solution  to  Problem  P. 

3.  If  x^  +  is  feasible  to  Problem  P,  then  the  search  vector  is  taken 
as  w^.  Otherwise,  is  taken  to  be  the  vector  of  maximum  length  along  w^ 
which  maintains  feasibility  of  x^  +  d ^ . 
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4,  Steps  3  and  4  of  the  algorithm  compute  the  step  size  taken  along  the 

vector  d^  in  order  to  form  As  proposed  by  Mukai  [10,  111 i  first  an 

estimate  of  the  step  size  is  calculated.  When  appropriate,  is  computed 

by  utilizing  a  quadratic  approximation  of  the  function  f  at  x.,  otherwise 

is  taken  equal  to  1.  In  order  to  ensure  feasibility  to  Problem  P,  the  first 

trial  step  size  ct.  used  in  conjunction  with  Armijo  line  search  [1],  is  the 

minimum  of  A.  and  1.  As  will  ba  shown  in  Section  4,  under  suitable  as sump- 
x  * 

tions,  for  large  i,  test  (2.5)  passes,  *  0,  and  ot  *  <  1.  This  confirms 

efficiency  of  the  line  search  scheme  where  eventually  only  one  trial  is 
needed  to  compute  the  step  size. 


3 .  Accumulation  Points  of  the  Algorithm 
Theorem  3.1  shows  that  each  accumulation  point  of  the  proposed  algorithm  is 
a  c-KT  point.  In  order  to  prove  this  theorem,  lemmas  3.1  and  3,2  are  needed. 
These  two  lemmas  extend  similar  results  of  Mukai  [10]  for  unconstrained  problems. 
In  order  to  facilitate  the  development  in  this  section,  the  following 
notation  is  used.  Let  w(x)  be  the  optimal  solution  to  Problem  D(x)  and  let 
8(x)  be  as  given  in  (2.3)  with  x_.  replaced  with  x.  Finally,  let  d(x)  *  3(x)v(x)  , 


Lemma  3.1 

Suppose  that  x  is  not  a  c-KT  point  for  Problem  P.  Ihen,  there  exist  scalars 
y  and  s  >  0  so  that  \i  <  a(x)  <  1  for  each  x  with  jjx-x  jj  <  s. 


Proof 

A  A 

There  exists  Sj^  >  0  so  that  I(x)  ■  I(x  )  for  all  jjx-x  jj  <  s^.  Thus,  the 

* 

feasible  region  for  Problem  E(x)  is  equal  to  that  of  Problem  D(x  )  for  all  x 
satisfying  [|x-x  |[  <  9^.  By  continuous  differentiability  of  f,  it  then  follows 
that  w(')  is  continuous  in  x  at  x  ,  Bee  for  example  Daniel  [3].  Particularly, 
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there  exists  a  number  >  0  such  that  I  (w(x>)  *  I  (w (x  ))  If  ||x-x  ||  <  s2> 

This  together  with  the  continuity  of  w(.0  and  the  formula  for  computing  $(•) 

£ 

imply  that  8(*)  is  continuous  in  x  at  x  .  Hence,  <!(•)  is  also  continuous. 

Since  x  is  net  a  c-KT  point,  then  w(x  )  #  0.  Furthermore,  -  a  Sc  >  c  if 

*  j  J 

a^w  >  0  which  implies  that  B(x  )  >  0.  Therefore  d{x  )  ^  0*  By  continuity 

of  6(0  and  d { * )  at  x  there  exist  scalars  q  and  s  >  0  so  that 


B(x)j]  d(x)  | 

;2  >f  e(x*)||d(x'v>ii2 

if  ||x-x*lj  <  s 

(3.1) 

f (x+£d(x) ) 

+  f(x-ed(x))  -  2f(x)  <  q 

^j# 

if  |jx-x  11  <  s 

(3.2) 

Now,  let  x  oe  such  that  ||x~x  j]  <  s.  Since  v(x)  solves  Problem  D(x),  then 
7f(x)  w(x)  <  -  |  2  j|w(x)|j2*  This,  in  turn,  implies  that  -  7f(x)td(x)  ~  z 
B(x)  |jd(x)  ||  and  from  (3.1)  we  get: 


-  ?f(x)td(x)  >  |  z  e(x*)  ||  d(x*)||2  =  y  >  0  (3.3) 

If  test  (2*5)  passes,  then  from  (3*2)  and  (3*3)  the  following  lower  bound  on 
X^  is  at  hand: 

i  =  -  £2Vf(x)td(x) _ i 

i  f(x-K;d(x))  +  f(x-£d(x)  -  2f(x)  —  q 

2 

Xf  test  (2.5)  fails,  then  X^  =«  1  and  hence  X^  min  { 1 ,  “  y.  Since 

0^  *  min  {l,  X  },  the  desired  result  follows. 


Lemma  3*2 
* 

If  x  is  not  a  c-KT  point  for  Problem  P,  then  there  exist  a  number  s  >  0  and 

& 

an  integer  m  so  that  k(x)  <  m  if  j|  x-x  ||  <  s,  where  k(x)  is  the  Armijo  integer 


given  by  (2.8)  with  x^  and  replaced  with  x  and  a(x)  respectively. 

Proof 

As  in  the  proof  of  Lemma  3*1  and  by  continuous  differentiability  of  f,  there 

•  *  \< 

exist  scalar s  s,  h,  and  v  >  0  so  that  for  ||  x-x  ||  <  s  the  following  hold: 

V&Cxjhfx)  <  -y  (3. A) 

(Vf  (x+gd(x))td(x)  -  \?f(x)td(x)|  <_  -|  y  for  each  gefOjh1  (3.5) 

Now  let  m  be  the  smallest  nonnegative  integer  so  that  (~)™  £  h  and  let  x  be 
such  that  |jx-x  [j  <  s.  Then  there  exists  9e[0,l]  such  that: 

f(x+(y)ma(x)d(x))  -  f  (x)  -  |(|)ma(x)7f(x)td(x) 

-  (y)ma (x) Vf (x+8  Cj) ma (x) d (x) ) td (x)  -  ~(i)ma(x)Vf (x)td(x) 

*  (|)ma(x)  {Vf(x+e(i)ma(x)d(x))td(x)  -  Vf(x)Cd(x)}  +  |  7f(x)td(x)"j  (3.6) 

Since  0(4)ma(x)  _<  h,  (3. A)  and  (3.5)  imply  that  the  right  hand  side  of  (3.6) 
is  <  0  which  in  turn  shows  that  k(x)  _<  m,  and  the  proof  is  complete. 

Theorem  3.1 

Either  the  algorithm  terminates  with  a  c-KT  point  for  Problem  P  or  else  gen~ 
erates  an  infinite  sequence  {x^}  of  which  any  accumulation  point  is  a  c-KT 


point  for  Problem  P* 
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Proof 

Clearly  the  algorithm  stops  at  x.^  only  If  x^  is  a  c-KT  point.  Now,  suppose 

that  the  algorithm  generates  the  infinite  sequence  jx^}.  Suppose  that  x  is 

K  * 

an  accumulation  point  so  that  x^  — *  x  for  some  infinite  set  K  of  positive 

K  * 

integers.  Since  f(x^)  is  decreasing  monotonically  and  since  f(x^)  ^  f(x  ) 

* 

then  fCx^}  — -*■  f(x  ).  Suppose  by  contradiction  to  the  desired  conclusion  that 
* 

x  is  not  a  c-KT  point.  From  Lemmas  3.1  and  3.2,  there  exist  positive  numbers 
y  and  y  and  an  integer  m  so  that  £  y,  7f(x^)td^  £  -  y ,  and  _<  m  for  large 
i  in  K.  Therefore, 

k . 

f(xi+1)  -  fCx^  < -|(|)  1  ctj  Vf(xi)td1  <  -  ~  py^)"1 

for  large  i  in  K.  This  implies  that  fCx^)  — ►  contradicting  the  fact  that 

fix)  — *  f(x  ).  This  completes  the  proof. 

4.  Eventual  Acceptance  of  the  Step  Size  Estimate 

* 

In  the  previous  section,  we  showed  that  an  accumulation  point  x  of  the 
sequence  {x^}  generated  by  the  algorithm  is  a  KT  point  to  the  perturbed  pro¬ 
blem  P*  given  below: 

? 1  :  minimize  f  (x) 

t  t  *  * 

subject  to  a^x  _<  a^x  for  j£l(x  ) 

a^x  £  bj  for  j£l(x  ) 

r  i  * 

Here,  we  assume  that  the  whole  sequence  converges  to  a  point  x  which 

satisfies  suitable  second  order  sufficiency  conditions.  Under  this  assump¬ 
tion,  we  show  that  test  (2.5)  is  eventually  passed.  Furthermore,  we  show 
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that  \ ^  <  1  and  that  k^  *  0  for  i  large  enough. 

The  second  order  condition  is  given  in  Definition  4,1.  It  is  well-known 

it 

that  x  satisfying  this  condition  Is  a  strong  local  minimum  for  problem  PT. 

£ 

That  is,  there  exists  a  number  y  >  0  so  that  f(x  )  <  f(x)  if  x  is  feasible 
to  problem  Pf  and  fjx-*x*jj  <  see  for  example  McCormick  [9]  and  Ran  and 
Mangasarian  [7]. 

Definition  4.1 

*  it  *  f  t  *  *  * 

Let  x  be  such  that  Ax  £  b  and  let  I(x  )  ’  {j:  a^x  >  bj  -  cj.  x  is  said 

to  satisfy  the  second  order  sufficiency  optimality  conditions  for  problem  P* 

it 

if  there  exist  scalars  £  0  for  jel(x  )  and  y  >  0  so  that: 

7f(x*)  +  l  *  u  a  «  0 
jel(x  )  J  J 

f(x  £  0,  a^d  £  0  for  jel(x  )>  ||  d  ||  *  1  »>  )d  >  y  (4.1) 

Theorem  4.1  shows  that  test  (2.5)  will  eventually  be  passed  so  that 
is  given  by  (2.6).  The  following  two  intermediate  results  are  needed  to 
prove  this  theorem. 

Lemma  4.1 

If  Cd  £  0  and  ||  d  j|  *  1  imply  that  dfcHd  £  y  >  0  then  there  is  a  number  8  >  0 
so  that  Cd  £  81  and  |[  d  ||  «  1  imply  that  dtHd  £  y/2. 

Proof 

Suppose  by  contradiction  that  for  each  integer  k  there  is  a  vector  such 


that 


(4.2) 


[[dk  |1  -  1,  Cdk  <  £  1,  and  d^Hdk  <  y/2 

Since  the  sequence  {d^}  is  bounded,  it  has  an  accumulation  point  d.  From 
(4.2),  |S d  |[  «  1,  Cd  £  0,  and  dtHd  £  y/2  which  contradicts  the  assumption  of 
the  lemma. 

Lemma  4.2 

If  either  {x^}  converges  or  {x:  Ax  £  b,  f(x)  f(x^)}  is  bounded,  then 

d±  +  0. 

Proof 

Since  0  <  6^  <_  1  and  *  B^w^,  it  suffices  to  prove  that  w^  -►  0.  Suppose 
there  exist  an  infinite  set  of  positive  integers  K  and  a  number  e  >  0  so 
that 


[[w^  11^.  £  for  leK  (4.3) 

Clearly,  under  either  of  the  assumptions  of  the  lemma,  there  exist  an  infinite  set 

^  C  K  and  a  point  x  so  that  x^  +  x  .  By  Theorem  3.1  x  is  a  c-KT  point 

_  . ,  *  * 
for  Problem  P.  Thus,  w  *  0  is  the  unique  optimal  solution  to  Problem  D(x  ). 

But  for  large  iefC* ,  l(x  )  »  I(x^)*  and  by  continuity  of  the  solutions  to 

D(*)  we  must  have  ||  | j  <  e/2  for  large  i  in  T.  This  contradicts  (4.3)  and 

the  proof  is  complete. 

Throughout  the  remainder  of  this  section,  the  following  notation  will 


be  used  for  any  scalar  y; 
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I 


hT  *  2  /  (1-y)  H(x,4rrd.)dy 
1  0  11 


C4.3) 


We  can  integrate  by  parts  to  obtain 


(4.4) 


f  (Xj+Yd j)  -  £(x±)  «  yVf  Ot^)  wd^  +  |  y2diRidi 


For  further  details,  the  reader  may  refer  to  Polak  [13,  p.  293]. 


Theorem  4.1 

r  i  * 

Let  {x^J  be  a  sequence  generated  by  the  algorithm.  Suppose  that  +  x  and 

* 

x  satisfies  the  second  order  optimality  conditions  for  problem  P  .  Then 
there  exists  an  integer  m  so  that  test  (2.5)  passes  for  all  i  >  in. 


Proof 


From  (4,3)  and  (4.4)  we  get: 


ffcj+edj)  -  f  (xi)  *  eVf(xi)td1  +  ~  e2<iiHidi 


f(x±-edi)  -  f(xA)  *  -eVf(xi)tdi  +  ~  e2djH~Gd4 


Adding  ve  obtain: 


f(x1+ed1)  +  fOx1-edi)  -  2f(Xl)  *  j  e2dJ(Hj+H”e)d1 


(4.5) 


★  £  *  k 

Now  for  jel(x  ),  a^x  >  b^  -  c.  Since  x^  x  then  for  i  large  enough, 

aV  >  b,  -  c  so  that  1el(x,)«  By  step  1  of  the  algorithm  a^w.  <  0  end  so 
Ji.j  1  ji— . 

t  *1  ,  A 

a .  ,  — ;;  <  0  for  i  large  enough  and  jel(x  ).  Likewise,  from  step  1  of  the 

j  II  di  11“ 


mmmm 
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t  t  1  * 

algorithm  Vf  (x^)  £  0  and  hence  Vf  (x^)  -n-j  ■  p  £  0.  Since  x^  x  , 

*  t  di  ^ 

any  number  0  >  0,  Vf  (x  )  <_  9  for  i  large  enough.  Thus,  Lemma 

II  ill 

the  second  order  conditions  imply  that 


d^H(x*)di  >  ||  2  for  large  i 


Nov  note  that 


||H^  -  H(x*)  ||  =  |1 2  /  (1-y)  jH(xi+yEd1)-H(x*)  dy 


<  2  /  (1-y)  ||H(x.+yed  )-H(x  )  ||dy 

0  ii 


* 

Since  ■*  x  ,  then  by  Lemma  4.2,  d^  -►  0.  Particularly,  for  i  large 
j|H(xi+yed1)-H(x*)  ||  <  ^  for  all  yE[0,l].  From  (4.7),  [|k^  -  H(x*)  ||  • 
This  together  with  (4.6)  yields: 


diHidi  *  diH(x*)di  +  dJ(H^-H(x*))di 

IMJI2  -  l|4t  II2  IIb'-ku*)  II 
1  4  lUi  II2  for  le.T%e  i 

Similarly, 

diHiGdi  -  4  11  di  11 2  for  large  i 
From  (4.5),  (4.8),  and  (4.9)  it  immediately  follows  that 


then  for 
4.1  and 

(4.6) 


(4.7) 

enough, 

.  I 
4  ' 


(4.8) 


(4.9) 
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f (Xj+edj)  +  fCx^edj)  -  2f(x1)  >  e2  ^\\d± 1(2  for  large  i  (4.10) 
From  C4.10),  if  test  (2.5)  fails  for  a  large  i,  we  must  have: 

e26t  ||d^  ]i 2  >  f(x1+edJ_)  +  ffc^-ed^  -  2f(xi)  >  e2  |||di  ||2 
that  is,  5  >  ^  .  If  the  conclusion  of  the  lemma  does  not  hold,  then  test 

V 

(2.5)  fails  infinitely  often  and  then  ^  0.  This  contradicts  >  4  for 

large  i,  and  the  proof  is  complete. 


Theorem  4.2 

Let  {x^}  be  a  sequence  generated  by  the  algorithm.  Suppose  that  x^  x  and 
that  x*  satisfies  the  second  order  optimality  conditions  for  Problem  P’.  Then 
there  exists  an  integer  m  so  that  f  (x^+a^d^)  -  f(xi)  <_  o^Vf^)  for  all 
i  ^  m,  that  is,  ^  »  0  for  all  1  >,  «. 


Proof 


By 


Theorem  (4.1),  test  (2.5)  passes  for  large  i  so  that  \±  is  given  by 


-  £27f(x1)tdi 


-  7f(x1)Cdi 

*  f  (xi+ed1)  +  f  (x^-ed^)  -  2f  (x^)  )d^ 


(4.11) 


If  X  •<_  1  so  that  -  X±,  then  from  (4.4)  and  (4.11)  we  get: 

fCXj-tOjij)  -  !<*,>  -  I  ai,f(*pt<li  ■  I  +  I  XlV,(l‘i)t<1l 


i  -  i  dJ(BE1+  H'jbdJ  -  JJ  <4-12> 
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it  ^4 

Since  xi  +  x  ,  then  by  Lemma  4.2,  d^  0.  Thus  H^1,  HE,  and  h”G  converge  to 
H(x*>  and  the  first  term  in  (4.12)  will  be  less  than  X2  Ijdjjj2  for  i  large 
enough.  As  in  the  proof  oE  Theorem  4.1,  d* (H^+H~£)d1  >_  ^  for  large  i. 

Substituting  in  (4.12),  the  desired  result  holds. 

Now  suppose  that  X^  >  1  so  that  0^*1.  Then 

-  f(Xl)  -  ~  a1Vf(xi)td1  -  \  djHjdi  +  |  Vf (x1)tdi  (4.13) 

Since  >  1*  then  from  (4.11)  we  must  have 

Vf(xi)tdi  <  -  |  d^(H^+R"e)di 

Substituting  in  (4.13)  we  get: 

f  (xi-hx1d1)  -  f(Xl)  -  |  c^Vf  (x1)tdi  <  \  [d^H^  -  i  dJ(H^e)di] 

-  yij  dJ(H^+H~E)d1  (4.14) 

That  the  right  hand  side  of  (4.14)  is  <_  0  for  large  i  follows  exactly  in  the 
same  manner  in  which  we  proved  that  (4.12)  is  £  0.  This  completes  the  proof. 

Finally,  we  state  certain  conditions  in  Theorem  4.3  below  which  guarantee 
that  Xj  <  1  90  that  “  X^  for  i  large  enough. 

Theorem  4.3 

Let  {x^}  ba  a  sequence  generated  by  the  algorithm.  Suppose  that  x^  -*■  x*  and 

* 

that  x  satisfies  the  second  order  optimality  conditions  for  Problem  P’ .  If 
*  <  ^  *  than  there  is  an  integer  m  so  that  Xj^  <  1  for  all  i  >  a,  that  is, 

•  Xj  for  all  i  _>  m. 
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Proof 

By  Theorem  4.1  there  is  an  integer  ic  so  that  for  i  >  a  we  have: 


-  e2Vf 

i  f(x1+€d1>  +  f (Xj-edj)  -  2f(x±) 


-  Vf(xi)tdi 


(4.15) 


As  in  the  proof  of  Theorem  4,1 

^  ^  [jd^l 2  for  i  large  enough  (4.16) 


Since  solves  Problem  then  there  exist  scalars  0  for  jel(x^) 

such  that 


7£ (Xj) 


+  zw^  + 


I 

jcl(xi) 


uijaj 


0 


(4.17) 


t 

u.  .a.w. 
ij  i  i 


».  o 


for  jelG^) 


(4.18) 


t  1  2 

From  (4,17)  and  (4,18)  it  follows  that  Vffx^)  *  -  z  j|wj|  .  But  by  Theorem  3.1 

x  is  a  c-KT  point  and  hence  the  optimal  solution  w  to  Problem  D(x  )  is 

*  4c 

w  »  0.  Since  x  ,  by  continuity  of  the  optimal  solution  to  Problem  IK*)> 

and  since  *  a^x^  >  c  for  each  jel+ (w^) ,  it  follows  from  (2.3)  that  6^  *  1 
for  large  1,  Thus  so  that 


Vf0ci)td^  =*  -  z  1 1  d | 2  for  large  i 


(4.19) 


Substituting  (4.19)  and  (4.16)  in  (4*15),  it  is  clear  that  <  1  for  i  large 
enough,  and  the  proof  is  complete. 
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